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Abstract 

A  two-eye  visual  environment  is  used  in  training  a  network  of  BCM  neurons.  We  study 
the  effect  of  misalignment,  betwe^^en  the  synaptic  density  functions  connecting  both  eyes  to  each 
single  neuron,  on  the  formation  of  orientation  selectivity  and  ocular  dominance.  The  visual 
environment  we  use  is  composed  of  natural  images.  We  show  that  for  the  BCM  rule  a  natural 
image  environment  with  binocular  cortical  misalignment  is  sufficient  for  producing  networks 
with  orientation  selective  cells  and  ocular  dominance  columns.  This  work  is  an  extension  of  our 
previous  single  cell  model  (Shouval  et  al.,  1996). 


1  Introduction 

It  is  generally  accepted  that  both  orientation  selectivity  and  ocular  dominance  of  receptive  fields  in 
the  visual  cortex  of  cats  are  dramatically  influenced  by  the  visual  environment  (for  a  comprehensive 
review  see,  Fregnac  and  Imbert,  1984).  Organization  of  the  different  properties  of  receptive  fields 
such  as  ocular  dominance  and  orientation  selectivity  across  the  visual  cortex  is  best  observed  by 
optical  imaging  techniques  (Bonhoeffer  and  Grinvald,  1991;  Blasdel,  1992).  To  the  best  of  our 
knowledge,  these  methods  have  not  been  used  yet  to  examine  how  receptive  field  organization  is 
influenced  by  changes  in  the  visual  environment.  It  has  been  shown  that  the  anatomical  correlates 
of  ocular  dominance  columns,  can  be  altered  by  changes  to  the  visual  environment  (Hubei  et  al., 
1977,  eg).  Furthermore  the  assumed  model  of  plasticity,  will  influence  the  structure  of  receptive 
field  organization  in  the  mature  cortex  (Erwin  et  al.,  1995). 

Different  models  attempting  to  explain  how  cortical  receptive  fields  evolve  have  been  proposed 
over  the  years  (von  der  Malsburg,  1973;  Nass  and  Cooper,  1975;  Perez  et  al.,  1975;  Sejnowski, 
1977;  Beienenstock  et  al.,  1982;  Linsker,  1986;  Miller,  1994).  Such  models  are  composed  of  several 
components;  the  exact  nature  of  the  learning  rule,  the  representation  of  the  visual  environment, 
and  the  architecture  of  the  network.  Most  of  these  models  assume  a  simplified  representation  of  the 
visual  environment  (e.g.  von  der  Malsburg,  1973),  or  replace  the  visual  environment  by  a  second 
order  correlation  function  (Miller,  1994). 
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Realistic  representations  of  the  visual  environment  have  only  very  recently  been  considered 
(Hancock  et  al.,  1992;  Law  and  Cooper,  1994;  Shouval  et  al.,  1996;  Olshausen  and  Field,  1996b), 
and  only  in  recent  years  have  the  statistics  of  natural  images  been  studied  and  used  for  predicting 
receptive  field  properties  (Field,  1987;  Field,  1989;  Atick  and  Redlich,  1992;  Bialek  et  al.,  1991; 
Liu  and  Shouval,  1994;  Shouval  and  Liu,  1996).  Once  actual  visual  scenes  are  used,  it  is  possible 
to  realistically  represent  two-eye  input,  and  account  for  the  fact  that  the  two  eyes  are  not  looking 
at  exactly  the  same  visual  scene.  For  example,  Li  and  Atick  (1994)  have  used  natural  images  to 
extract  detailed  two-eye  power  spectra  from  stereo  images.  They  have  used  these  results  to  predict 
properties  of  cortical  receptive  fields. 

We  have  recently  shown(Shouval  et  al.,  1996)  that  single  cell  BCM  neurons,  trained  in  a  binoc¬ 
ular  natural  image  environment  can  develop  both  orientation  selectivity  and  varying  degrees  of 
ocular  dominance.  We  have  also  shown  in  that  study  that  PCA(Oja,  1982)  neurons  can  not  de¬ 
velop  ocular  dominance  as  a  result  of  the  invariance  of  the  two-eye  correlation  function  to  a  two-eye 
parity  transformation. 

In  this  paper  we  extend  this  study  to  networks  of  interconnected  neurons.  The  network  inter¬ 
actions  have  two  types  of  affects: 

(a)  They  can  alter  properties  of  single  cell  receptive  fields. 

(b)  They  produce  an  organization  of  receptive  fields  across  the  cortex. 

We  examine  here  these  two  types  of  effects. 

Unlike  our  previous  study,  we  do  not  present  results  using  the  PCA  rule  since  in  that  case  the 
network  does  not  qualitatively  affect  the  single  neuron  results  and  in  particular  aU  cells  remain 
binocular.  This  result  can  be  shown  to  foUow  from  an  extension  of  the  two-eye  parity  method  to 
linear  networks. 

Recently  several  models  that  attempt  to  generate  sparse  coding  have  been  proposed  for  learning 
in  early  visual  cortex  (Foldiak,  1990;  Fyfe  and  Baddeley,  1995;  Olshausen  and  Field,  1996b).  In 
this  paper  we  demonstrate  that  BCM  neurons  achieve  sparse  coding  (Intrator  and  Cooper,  1992) 
without  explicitly  attempting  to  maximize  the  spareness. 

2  Details  of  the  model 

The  BCM  theory  (Beienenstock  et  al.,  1982)  was  introduced  to  account  for  the  striking  dependence 
of  the  sharpness  of  orientation  selectivity  on  the  visual  environment.  We  have  used  a  variation, 
due  to  Intrator  and  Cooper  (1992),  for  a  nonlinear  neuron  with  a  non-symmetric  sigmoidal  transfer 
function.  Using  the  above  notation,  synaptic  modification  is  given  by: 

mj(x)  =  Tj<^(c(x),  0iw(x))dj,  (1) 

where  the  neuronal  activity  is  given  by  c,  <^(c(x),  Qm(x))  =  c(x)  (c(x)  -  0a/(x)),  x  denotes  the 
coordinates  of  the  neuron  within  the  network,  mj  are  the  synaptic  weights,  dj  the  inputs  and  0m  is 
the  modification  threshold.  The  modification  threshold,  Qm,  is  a  nonlinear  function  of  some  time 
averaged  measure  of  cell  activity,  given  by 

0m(x)  =  £[c^(x)],  (2) 

where  E  denotes  the  expectation  over  the  visual  environment. 
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Binocular  Model 


^  t  Visual  pathway 


Figure  1:  Schematic  diagram  of  the  two  eye  model,  including  the  visual  input  preprocessing.  Dif¬ 
ferent  cells  in  this  network  receive  the  same  partially  overlapping  inputs. 

In  the  lateral  inhibition  network  c  is  also  a  function  of  the  activity  of  it’s  neighbors.  The  activity 
is  given  by^ 


c(x)  =  a  1^53  “  x')(T(m(a;')  •  d)  j  (3) 

Where  I  is  the  lateral  interaction  matrix  for  which  we  used  a  balanced  DOG  and  is  given  by 
/(x)  =  (l/27r4)exp(-i  )  -  (l/27r(7f)exp(-^  ) 

Where  oe  aud  <7/  are  the  length  scales  of  the  excitation  and  inhibition  respectively.  The  transfer 
function  a  is  non  symmetric  around  0  to  account  for  the  fact  that  cortical  neurons  show  a  low 
spontaneous  activity,  and  can  thus  fire  at  a  much  higher  rate  relative  to  the  spontaneous  rate,  but 
can  go  only  slightly  below  it^. 

We  have  used  the  same  visual  environment  that  we  have  described  in  detail  in  our  single  cell 
study  (Shouval  et  al.,  1996).  It  is  composed  of  a  set  of  24  natural  images  scanned  at  a  256  X  256 
pixel  resolution.  We  have  avoided  man-made  objects,  because  they  have  many  sharp  edges,  and 
straight  lines,  which  make  it  easier  to  achieve  oriented  receptive  fields.  We  have  modeled  the  effect 
of  the  center  surround  retinal  and  LGN  projections,  by  convolving  the  images  with  a  difference 
of  Gaussians  (DOG)  filter,  with  a  center  radius  of  one  pixel  (cri  =  1.0)  and  a  surround  radius  of 
three  (cr2  =  3)  As  illustrated  in  Figure  1,  the  input  vectors  from  both  eyes  are  chosen  as  small, 

^We  have  also  used  the  type  of  lateral  network  described  in  Intrator  and  Cooper  (1992)  with  the  same  7  and  found 
no  qualitative  difference  in  the  results. 

^The  actual  sigma  used  in  the  simulations  is  (e®  —  e~®)/(0.05e®  -f  5e“^). 
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partially  overlapping,  circular  regions  of  the  preprocessed  natural  images;  these  converge  on  the 
same  cortical  cell. 

3  Results 


Figure  2:  The  top  left  figure  represents  combined  Ocular  Dominance  and  Orientation  maps  ex¬ 
tracted  by  a  network  of  24  by  24  neurons  with  receptive  fields  of  diameter  20  and  a  shift  of  15 
pixels  between  the  two  eyes.  For  this  network  cr^;  =  1  and  aj  =  5.  The  orientation  of  the  bars 
codes  for  the  preferred  orientation  of  the  cells,  it’s  length  represents  the  degree  of  orientation  selec¬ 
tivity,  the  gray  scale  of  the  background  represents  Ocular  Dominance.  On  the  right  is  a  histogram 
of  the  activity  of  all  neurons  in  the  network  in  response  to  40  patterns  chosen  from  the  data  set. 
We  can  see  that  the  response  is  sparse;  most  neurons  have  a  large  response  only  a  small  fraction 
of  the  time.  On  the  bottom  the  feed  forward  receptive  fields  from  the  4  by  4  region  of  the  network 
that  is  enclosed  in  the  black  box,  are  displayed.  On  the  bottom  right  the  connections  to  the  right 
eye  (m’’)  and  on  the  bottom  left  the  connections  to  the  left  eye  {vn}). 

Figure  3  depicts  results  of  a  typical  network.  The  receptive  fields  displayed  in  such  networks  axe 
very  similar  to  those  obtained  for  single  cells  (Shouval  et  al.,  1996;  Law  and  Cooper,  1994).  They 
are  orientation  selective  to  all  orientations  and  show  various  degrees  of  ocular  dominance.  The 
degree  of  ocular  dominance  depends  on  the  overlap  between  the  receptive  fields  of  the  two  eyes, 
larger  overlaps  producing  more  binocular  cells. 

The  activity  histogram  displayed  in  the  upper  right  hand  side  of  figure  3  shows  that  the  activity 
of  neurons  in  the  network  is  sparse. 
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The  organization  of  such  receptive  fields  across  the  cortex,  that  is  the  organization  of  Ocular 
Donainance  bands,  Iso- Orientation  bands  and  the  relationship  between  them  depends  crucially  on 
the  details  of  the  lateral  interaction  I,  as  can  be  seen  in  figure  3.  It  is  possible  to  change  the 
parameters  of  the  lateral  interaction  in  order  to  achieve  cortical  maps  which  exhibit  a  greater 
or  a  smaller  resemblance  to  experimental  cortical  maps.  However  we  do  not  attempt  to  explore 
that  here,  since  we  believe  that  cortical  maps  are  influenced  by  many  other  factors  such  as  innate 
preferences,  the  more  complex  three  dimensional  anatomy  of  the  cortex,  and  the  details  of  the 
imaging  techniques  used  to  obtain  the  maps. 


Figure  3:  The  effect  of  changing  the  parameters  on  network  organization.  On  the  left  the  effect  of 
changing  the  overlap  between  the  eyes  is  displayed.  The  displacement  between  the  two  eyes  is  5 
pixels,  apart  from  that  the  parameters  are  identical  to  the  one  in  figure  3.  The  Ocular  Dominance 
bands  become  much  weaker  in  this  case.  On  the  right  the  effect  of  changing  the  lateral  interaction 
term  I  is  displayed,  here  (tb  =  0.7  and  cr/  =  2,  apart  from  that  the  parameters  are  identical  to  the 
one  in  figure  3.  This  has  the  effect  of  greatly  reducing  the  continuity  in  the  orientation  and  Ocular 
Dominance. 


4  Discussion 

We  have  extended  our  single  ceU  results  (Shouval  et  al.,  1996)  to  networks  of  connected  neurons. 
The  networks  have  been  trained  with  a  natural  scene  environment  that  was  preprocessed  with  a 
DOG  filter.  The  two  eyes  were  exposed  to  small  partially  overlapping  portions  of  these  images. 
Like  in  our  single  ceU  study  we  have  shown  that  orientation  selective  ceUs  with  varying  degrees  of 
ocular  dominance  develop.  The  degree  of  ocular  dominance  depends  mostly  on  the  overlap  between 
the  inputs. 

Sparse  neuronal  representation  can  be  roughly  characterized  by  neuronal  activity  which  is  inac¬ 
tive  most  of  the  time,  namely  has  a  distribution  of  activity  which  is  highly  peaked  at  zero  and  thus 
has  heavy  tails.  For  this  reason  sparse  activity  is  associated  with  kurtotic  activity  distribution.  It 
is  thus  clear  that  various  types  of  distributions,  with  the  common  property  of  having  large  peak  at 
zero,  wiU  fit  into  this  definition.  The  instantaneous  sum  of  activity  of  a  network  of  sparse  neurons 
wiU  have  a  Gaussian  distribution  if  the  neurons  are  independent  (although  each  of  them  may  be 
highly  kurtotic)  and  will  maintain  its  kurtoic  shape  if  the  code  is  also  non  redundant.  In  the  case  of 
n  linearly  independent  clusters  in  an  n  dimensional  space,  it  has  been  shown  (Intrator  and  Cooper, 
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1992)  that  a  network  of  n  BCM  neurons  will  find  a  solution  so  that  the  weight  vector  of  each 
neuron  is  orthogonal  to  aU  but  one  of  the  cluster  centers.  In  other  words,  if  cluster  i  appears  with 
probability  pi,  the  neuron  that  becomes  selective  to  this  cluster  is  inactive  with  probability  1  -  p,-. 
Furthermore,  when  it  is  active,  the  neuronal  activity  C,-  is  proportional  to  l/p*.  Thus  the  sum  of 
n  such  neurons  -  C,  has  a  distribution  in  which  the  tall  decays  as  1/C  (Figure  3,  right).  This  is 
certainly  heavier  than  the  tail  of  a  Gaussian  distribution  which  decays  as  exp(-C'^). 

We  emphasize  that  in  the  BCM  network  case,  sparse  coding  is  an  outcome  of  the  dynamics 
of  the  BCM  learning  and  is  not  specifically  built  into  the  code  generation  process  as  is  done  in 
(Olshausen  and  Field,  1996a).  Furthermore,  in  accordance  with  Barlows  predictions  (Barlow  and 
Pettingrew,  1971),  events  that  occur  with  high  probability  are  conveyed  by  a  less  active  neuron 
than  events  which  occur  less  frequently.  Such  events  will  be  conveyed  by  a  neuron  that  is  quiet 
most  of  the  time,  but  fires  strongly  when  the  event  is  detected.  It  will  be  interesting  to  compare 
the  resulting  code  with  methods  that  maximize  sparsity  or  kurtosis  as  a  goal  for  neuronal  coding 
and  feature  detection  (Fdldiak,  1990;  Fyfe  and  Baddeley,  1995;  Olshausen  and  Field,  1996b). 

The  organization  of  receptive  fields  across  the  cortex,  presented  above,  includes  many  of  the 
components  required  from  a  cortical  map.  It  has  both  ocular  dominance  columns  and  varying 
orientation  preference.  The  orientation  map  has  both  linear  regions  and  non-linear  regions  in  which 
sharp  transitions  in  orientation  preference  occur.  The  organization  of  receptive  fields  across  the 
cortex  depends  critically  on  the  parameters  of  the  lateral  interaction  term,  but  we  believe  that  in  the 
real  visual  cortex,  cortical  maps  are  influenced  as  well  by  other  factors  such  as  innate  preferences, 
the  more  complex  three  dimensional  anatomy  of  the  cortex  and  the  shape  of  the  boundaries  of  the 
cortical  area. 

As  far  as  we  know  this  is  the  first  network  model,  trained  in  a  natural  image  environment,  which 
develops  concurrently  both  ocular  dominance  and  orientation  selectivity.  The  model  presented 
recently  by  Olshausen  and  Field  (96)  develops  orientation  selective  neurons  from  a  natural  image 
environment,  however  it  does  not  attempt  to  model  the  binocular  aspects  of  cortical  cells  or  the 
organization  of  these  receptive  fields  across  the  cortical  sheet. 
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